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ABSTRACT 

This study attempts to determine if the average 
American junior college library annual report is used as an 
evaluation tool. If so, it should demonstrate how well library 
objectives are being met. A survey of a random sample of 260 junior 
college libraries collected information regarding annual reports, 
written statements of objectives, and evaluation consciousness. Some 
findings based on the 170 responses { 65 %) were; (1) 15 % of the 
respondents prepare annual reports; (2) about one-half of the annual 
reports are prepared by libraries that have no written objectives; 

(3) more than one-half of all annual reports are prepared without 
reference to objectives; (4) collection and circulation data are the 
dominant topics in the annual reports; (5) one-half of the 
respondents use two or more methods for evaluation, and were 
therefore classified as "evaluation conscious"; and (6) high 
evaluation consciousness is associated with annual report 
preparation. It was concluded that to write more meaningful annual 
reports, librarians must search for clear and specific evaluation 
criteria and also present the accomplishment of their libraries' 
services in terms of predetermined, operationally defined objectives. 
(RN) 
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ABSTRACT 



If evaluation of a library means the determination of the degree to 
which the library'a objectivea are fulfilled, then the annual report sub* 
nittad by the average Junior college librarian to hla administrative 
superiors is not a good evaluation tool. Annual reports, it was found, 
often present only those library conditions that are easiest to aeasure 
and bear little relationship to objectives, expressed or implied. To write 
more meaningful annual reports librarians must (1) search for clear and 
specific evaluation criteria, and (2) present the accomplishment of their 
libraries' services in terms of predetermined, operationally defined 
objectives. 



INTRODUCTION 

The "Standards for Junior College Libraries" published in 1960 by 
the Association for College and Research Libraries contained a clause 
suggesting that an annual report to the administrative officers of the college 
would be useful to bring out "the accomplishments of library service". (1, p.201) 
The draft of the' new "AAJC-ACRL Guidelines for Tvo*Year College Library 
Learning Resource Centers" carries this theme over, indicating that an annual 
report is "essential" to inform managasmnt of library activities. (2, p.269) 

If, in the words of the "Standards", the annual report is to present 
"the accomplishments of llbrury service" it is obviously necessary that the 
library have accomplished something during the reporting period. And surely, 
any accomplishment worth reporting must be related to the functions or 
objectives of the llbirary. To extend this train of thought, one can say 
that anything that relates to the functions or objectives of the library is 
subject matter worth reporting to the administrative officers of the college. 
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Now, when one considers that the word "accomplish" has a broad range of 

meanings from "bring to full success" to "complete", "fulfill", or just " 

"perform", it is quite logical to re-deflne the annual report more precisely 
as "a document that reports on the extent to which the library fulfills its 
functions or objectives". It is in this sense that the annual report la seen 
here as an evaluation tool. 

If the paucity of library literature on the topic la a valid indicator, 

however, very little is known about the annual report in the junior college 

v 

library. In order to gain a clearer picture a national sample of 260 junior 
college libraries, randomly selected from those listed In American Junior 
Colleges, Seventh Edition, were surveyed. Of these, 53 were In large colleges 
(2000 PTE enrollment or more). The remaining 207 libraries were In small 
institutions (less than 2000 PTE enrollment). The survey was conducted by 
meana of a checklist (Appendix A). Of the 260 Institutions contacted, some 
30 had either been cloaed or converted to four-year schools. The remaining 
60 non- respondents did not answer follow-up notices and could not be reached 
before the deadline for this report. A total of 170 usable responses were 
received. 

The data obtained from the returned checklists were used to answer the 
following questions: 

1. How many junior college Ubrarlea prepare annual reports? 

2. How many have a written statement of objectives? 

3. What la the relatlonahlp between the annual report and the 
atatement of objectives? 

4. What Is the composition of the annual reports? 

3. How do junior college libraries rate on evaluation consciousness? 

6. What la the relationship between this rating and annual report 
preparation? 
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Question i 

In answer to the first question, "How many junior college libraries 
prepare annual reports?", it was found that 129, or 75 per cent of the 
respondents, stated that they prepared annual reports. In the absence of 
norms it can be assumed that this is about as could be expected: three out 

of four libraries meet the requirements of the standards. Since the returns 
were incomplete it is dangerous to extend the findings from the sample to 
the entire population. In answering a survey inquiry such as this, a librarian 
who for one reason or another finds himself on the aide of those who do not 
meet the standards in respect to the annual report, might hesitate to admit 
this fact. Rather than prevaricate, he might conceivably procrastinate, and 
in the end simply fail to respond. It is probably safe to estimate that half 
of the non-responses are due to this cause. This information can be used to 
adjust the population estimate downward as follows: 

Number of respondents 
Number of non- respondents 
Total number of active libraries in sample 



170 

60 

230 



Actual "yes" answers 129 (75Z of 170) 

Potential "yes" answers among 

non-respondents 30 

159 (69Z of 230) 



The sise of the sample was chosen for a reliability of +5 per cent at 
the 95 per cent confidence level. In view of the limitations of the data, 
one might want to be pessimistic about the inferences and state with 95 per cent 
confidence that it seems that about two thirds of all Aawrican junior college 
libraries prepare annual reports as suggested in the "Standards". 
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Question:) 2 and 3 



In order to discuss the value of the annual report as a library evalu- 
ation, tool, it will only be necessary to examine the objectives of the library 
and match them with corresponding statements in the annual report. For an 
annual report to be completely satisfactory as an evaluation tool, one would 
assume that it should answer fully whether the library has accomplished its 
objectives, item for item. For every specific library objective there should 
be a corresponding statement in the annual report. Schematically, this can 
be shown as in Table 1. 



TABLE 1 

Model of an Ideal Evaluative Annual Report 



Objectives 


Annual Report Statements 


The library should. . . 


The library . . . 


. . . do X 


. . . did X 


... do Y 


... do not do Y 


... do Z 


. ... did sosc Z 



To carry this analysis through, a list of objectives must be found for 
each library studied. As a first step, an effort was made to determine the 
number of junior college libraries that have written statements of objectives. 

Of the 170 respondents, 77 said they had such statements. Sixty-five of these 
also said they prepared annual reports. These data say, on the face, that hslf 
of the annual reports are prepared by libraries that have no written objectives. 
This situation is visualised in Table 2. 
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TABLE 2 

American Junior College Libraries Categorized 
by Presence or Absence of 
Annual Reports and Statements of Objectives 







Have 

Objectives 


Do not have 
Objectivea 


Row 

Totals 


Prepare annual 


reports 


65 


64 


129 


Do not prepare 


annual reports 


12 


29 


41 


Column totals 




77 


93 


170 



As a second step, the respondents were asked to submit copies of the 
statements of objectives they said they prepared. Of the 77, only 3 did 
submit a copy. This negligibly small number of actual statements submitted 
suggests very strongly that the reported figure of 77 is probably a good deal 
higher than reality warrants. If all vague statements of the kind generally 
found in college catalogs, such as "the library has X number of volumes to 
assist students in research and reference reading", were excluded it would 
probably be found that a good deal more than half of all annual reports are 
prepared without reference to clearly defined objectives. This is all ths 
more surprising if one considers that in 1963 the ACRL Standards and Criteria 
Committee recommended that Junior college libraries develop codas of library 
policies, among them "an outline of the functions [and) objectivas of the 
library" (3, p.502), and that the "Guidelines"' of 1971 continue in this vein by 
demanding for every library "a statement of defined. . .objectives" (2, p.267). 

As a result of these findings, the attempt to itemise the objectives 
of the various libraries had to be abandoned. 
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Question 4 

To obtain data concerning the composition of the annual report, it was 
necessary to solicit copies of the annual reports the respondents stated they 
prepared. Only 29 of the 129 positive respondents, or 22 per cent, actually 
sent copies. While this Is a more encouraging yield than that for statements 
of objectives, It Is still a meager return. The fourth question, therefore, 

t 

can be answered only In a limited sense on the basis of the composition of 
the 29 reports received. 

These reports were found to come In great variety of patterns and styles. 
Some wars short, some lengthy; some were In narrative style, soma consisted of 
tabular material, some combined both o£ these features; some presented data 
In large categories, others broke them down Into minute detail. The only 
unifying feature that could be Isolated was that they were all typewritten on 
8 1/2 by 11 Inch paper. To compare the 29 reports with each other, therefore, 

the line of typescript was taken as the countable unit. In this way It was 

possible to calculate the percentage of lines devoted to particular topics. 

In all, nine distinct evaluative topics were Isolated. The results of this 

analysis are shown In Table 3. 



TABLE 3 

Topics of Annual Reports, In Descending Order by Amount of Space 
Devoted to them by 29 American Junior College Libraries (In per cent 
of lines of typescript) 



Topic 



Median Range 



Collection data, such as site, growth, and composition 16.0 100.0 

of the book collection 

Circulation data, such as horns and room use, number of 

lnterllbrary loan transactions, opening hours 14.4 50.0 
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Non- evaluative data 


12.0 


50.0 


Audio-visual data, such as media and equipment 
acquisition and usage 


8.0* 


55.0 


Personnel data, such as staff sine 


3.2 


23.4 


Facilities data, such as floor and work space 


2.4 


35.0 


Technical services data 


2.0 


19.8 


Financial data 


1.4 


50.0 


Reference service data, including instruction 


b 


50.0 


User relations data, such as information on 
library coaaslttee 


b 


4.2 



* about half of the statements identified as audlo-vlaual data were 
concerned with acquisitions and usage of media and equipment. 

^ less than one. 

Inspection of Table 3 suggests that in terms of percentage of space 
devoted to them, two topics contribute the greater part of the evaluative 
information to the average junior college library annual report: collection 

data and circulation data. To be on the safe side, significance of this 
observed imbalance among the topics was tested on the model of no difference 
in the median percentage of lines devoted to the ten different classes, it 
was hypothesised, was merely a chance deviation from a rectangular population. 
The rejection region was set at ■ 0.01. Using the chi square test, the 
value obtained was X* • 54.4. For nine degrees of freedom this value has a 
probability of occurence under the null hypothesis of no difference of 
p o.OOl. The null hypothesis was rejected, which strengthens the con- 
clusion that collection and circulation data are Indeed the dominant topics 
in the annual reports of American Junior college libraries. 

In view of the high percentage of collection data, an attempt was made 
to segregate information on collection else and growth from information on 
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collection quality. If a report devoted at lcaat one line of typescript to 
the question of collection composition* for example* this was counted as an 
instance of collection quality reporting. Table 4 shows the result of this 
segregation effort. Emphasis appears to be on quantitative collection measure. 

TABLE 4 

Collection Data Categorized by Aspects 
(Number of Libraries Reporting) 



Aspects 



Number of Libraries 
Reporting Such Data 



Collection size 


21 


of 


29 


Collection growth 


22 


of 


29 


Collection quality 


8 


of 


29 


Other aspects than those 


6 


of 


29 


accounted for above 









In another study, the annual reports of 13 California junior college 
libraries were analysed by topics covered (4). The results of that analysis 
are given In Table 5 for comparison. 



TABLE 5 

Topics of Annual Reports Arranged In Descending Order 
by amoung of space devoted to them by thirteen California 
junior college library annual reports (Per cent of sentences) 



1. Holdings and acquisitions 20 

2. Circulation and use 8 

3. Expenditures and budget 7 

4. Facilities and equlpnant 6 

5. Personnel 5 

6. Orientation 3 
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7. Acquisition and cataloging processes 

8 . Opening hours 



3 

1 



9. Inter library loans 

10. Photocopy services 

11. Reference services 



a 

a 

a 

a 



12. Book lists and similar user relations 



• less than ona. 

In this study, audiovisual data vara 1 usped together with book data. Other* 
wise the two tables (Tables 3 and 5) are quite comparable. On the strength 
of these two Independent sets of observations there Is good reason to believe, 
then, that collection slse and circulation data are by far the meat dominant 
topics In the Annual Reporta of American Junior college libraries; that 
personnel, facilities, technical procasslng, and fl nanclal data are 
significantly less prominent; and that data concerning reference service and 
user relations are practically absent from such reports. 



An answer to the fifth question , "How do Junior college libraries rate 
on a scale of evaluation consciousness?", required the establishment of such 
a scale. It was reasonsed that In a library where the administration was 
conscious of the Importance of evaluation, many methods would be employed 
In the course of a five year period to carry out such evaluation tasks. 
Likewise, a library that was totally unconcerned with evaluation would not 
carry out any evaluation studies during the same period. The number of 
different types of evaluation studies carried out In the most recent 
five year period, therefore, was adopted as a measure of a library's 
evaluation consdousnesa . The Checklist (Appendix A) presented five 
options of specific evaluation mathoda. Space was left for write Ins. To 



Questions 5 and 6 
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prevent contentful t ion of the dete by lrrelevent information, eeperete answer 
blocks were provided for eccredltetion surveys end the accumulation of 
service statistics, two tasks presumably carried out by all Ubrarlee. Ihe 
results of this rating process are given in Table 6. 

TABLE 6 

Humber of Asnricen Junior college libraries grouped by 
number of different evaluation methods used in the laet 
five yaaro. 



Number of Methods 


Number of Libraries 


Cumulative Nisnber 


0 


29 


170 


1 


58 


145 


2 


48 


85 


3 


25 


37 


4 


10 


12 


5 


2 


2 



As Table 6 shows, half of the respondents used two or more methods. This 
group represents the "evaluation conscious" libraries. The rest are catego- 
rlsed as "not evaluation conscious". The complete checklist returns on 
evaluation methods are given In Appendix B. 

The last question, concerning the relationship of evaluation con- 
sciousness rating to presence or absence of annuel reports, had for Its 
purpose to test the hypothesis that libraries that prepare annual reports 
are the same ones that rate high on evaluation consciousness. If this relation- 
ship could be shown to exist it would strengthen the view that the annual 
report Is more than a presentation of library statistics but Is Intended as 
an evaluation instruaMnt. The data are shown in Table 7. 
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TABLE 7 

Aaerlcan Junior eollogo librarian dlatr&butad Into iroupi by 
preparation or non-preparation of annual raporta and by tuo 
lava la of avaluatlon eonaciouaaaoa (Nunber of librarian) 





Evaluation Gonoclouanaoo 


Bow 




High 


Low 


Totala 


Prepare annual raporta 


74 


S3 


129 


Do not 

Prepare annual raporta 


12 


29 


41 


Colunn totala 


86 


84 


170 



Tabla 7 auggeate that high avaluatlon eonaclouanaaa la lndaad aaaoelatad with 
annual raport praparatlon. To naaaure tha extend of thin aaaoclatlon tha 
contlngancy coafflelant wan calculated for tha data of Tabla 7. Thla valua 
cane to C ■ 0.23; tha valua of chi aquara for thaaa data wan 9.3. for oua 
dagraa of fraadon thin van found to ba algnlf leant beyond tha 0.01 level. 
While tha aaaoelatlon uaa not vary atrong, It aaau that tha aaaunptlon la 
atlll Juatlfled that librarian preparing annual raporta do no with tha 
Intent of producing an avaluatlve raport. 
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Conclusions 

Ttio dots collected for this study suggest that annual reports are 
prepared In the majority of American Junior college libraries are written 
by people that are highly conscious of the Importunes of library evaluation. 
These annual reports can therefore be regarded as Instruments Intended to 
present "the accomplishment of library service". 

The conclusions concerning the performance of the average American 
Junior college library annual report as an evaluation tool* however, are 
less positive. As was suggested above, evaluation of a library, If anything, 
must naan the detsnalnatlon of the degree to which the library's objectives 
are fulfilled. The study showed that considerably more then half of all 
annuel reports are prepared In libraries that do not even have written 

i 

statements of objectives. Surely, such annual reports cannot possibly have 
any relationship to well defined library objectives. 

Many will argue that after a hundred years of successful practice, 
American librarians will surely know how to tell a good library from a bad 
one; that It Is absurd to say that libraries have no objectives. 

Can the annual report be said to measure "Implied" objectives, than, 
less wall defined and unwritten? If this were granted one should expect 
that for each "Implied" objective there should be a section In the annual 
report corraepondlng In length and emphasis to the relative importance of 
the implied objective. The data of this study do not support this Inter- 
pretation, either. For example, as the new "guidelines" specify, an 
Important part of Junior college library service Is the assistance that 
students receive "in smatlng their needs, articulated or unexpressed" (2, 
p. 277). Thera cannot be a Junior college librarian alive today who would 
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not NhoUhiartidly subscribe to the view that reference service is of prims 
importance in ths junior col legs library, and that guidance, orientation, and 
personalised library instruction art among the top objectives of the library. 
Also, facilities are certainly no minor matter in collage libraries. The 
Dartmouth user study of a few years ago showed that 63 per cent of the people 
in the library at a given moment ware there not because they needed to utilise 
the collection but because they needed a place to study (5, p.406). Robert 
Sonmer's study of 1,563 college and Junior college students showed that half 
had come to the library because it was a quiet place to study and was con* 
venlently located; only 27 per cent said they had come to the library because 
of the availability of books (6, p.251). Kenneth Allen found that 61 per cent 
of the students questioned in his study had come to the library to use the 
facilities, not the collection (3, p.7l). Yet, as the data in Table 3 have 
shown, the amount of space devoted to these topics in annual reports is 
almost insignificant when compared to the amount of space devoted to the 
collection and circulation data. One very plausible if not pleasing 
explanation for this unbalanced state of affairs is simply that the annual 
report, at this time, is not primarily a report on the fulfillment of 
library objectives, expressed or lnplled, but rather a presentation of those 
library conditions that era easiest to measure. 

Considering the limitations of the saaple of cases studied, one oust 
be careful in drawing inferences. However, the results of this study support 
Burners' recent observation that library annual reports are often based on 
"superficial token objectives ... that do not actually guide or further 
actions" (7, p.63). It is the conclusion of this study that the average 
American Junior college library annual report cannot be relied upon to 
denonstrate conclusively how well the objectives of the library are being 
met. It is not a true evaluation tool. 
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Racoanandatlctna 

If further prograaa la to ba wade In tha art of writing awanlngful 
avaluatlvt annual raporta , aora attantlon nuat ba paid to tha objactlvaa 
and functiona of tha Junior collaga library. 

Va Ubrarlana have hara an opportunity to advanea tha undaratandlng 
of tha thaoratlcal aapacta of tha profaaalon by focualng attantlon on a 
fundamental probloai of library aelanea: tha quaat for clear, apaclflc, 

oparatlonally daflnad objactlvaa, and for valid and rallabla crltarla or 
■aaauraa by which to evaluate prograaa. 
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Appendix A 

LIBRARY EVALUATION METHODS CHECKLIST 



Please check the appropriate box if any of the evaluation net hods 
listed below have been used in your library in the last five years: 

(1) Accreditation survey ( ) 

(2) Conprehensive library survey conducted by outside 

agency other than accreditation tean ( ) 

(3) Conprehensive library self survey ( ) 

(4) Library use study or user poll, not included 

in 1 through 3 above ( ) 

(5) Cost or cost-effectiveness study, not included 

in 1 through 4 above ( ) 

(6) Collection evaluation study (such as checking your 

holdings against a published list of recomsended books) ( ) 

(7) Conpilation of service records such as circulation, 

Acquisition and processing statistics, not included in 

1 through 6 above ( ) 

(8) Other nethods not included in 1 through 7 above ( ) 

Also please indicate by a check in the appropriate box if ... 

. . .you prepare an ANNUAL REPORT to the administrative officers 

of the college ( ) 

...there exists in your institution a written statement of 

LIBRARY OBJECTIVES ( ) 

. . .you have a written ACQUISITIONS POLICY ( ) 

Thank you for your time. Please return this checklist in the enclosed 
envelope to H. Hoffman, Catalog Librarian, Santa Ana College, Santa Ana, 

CA 92707. 



Appendix B 



The 
llbrerlee 
have used 
( 2 ) 
O) 

(4) 

(5) 

( 6 ) 



Library Evaluation Methods Checklist mailed to 260 junior college 
listed five distinct options of evaluation methods a library may 
In the most recent five year period: 

Comprehensive library survey conducted by an outside agency 

Comprehend ve library self survey 

Library use study or user poll 

Cost or cost-effectiveness study 

Collection evaluation study 



When these amthods were rank ordared by the number of libraries that 
raported having used them, the following table resulted. 

Method Huafrer of Libraries Per Cent of Respondents 



Collection 


115 


79 


Comprehensive self survey 


65 


43 


Library use study 


58 


39 


Comprehensive outside survey 


15 


10 


Cost and cost-effectiveness study 


12 


9 


Miscellaneous others 


4 


3 



Mote: Methods (1) Accreditation survey; and (7) Compilation of service 

records were Included only to filter out those methods that every library 
could ba expected to have been using. This would have distorted the data. 
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